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enough for melody retrieval services over a WWW 
network. 



[0012] In a further preferred aspect of the invention 
there is provided a media retrieval system which can re- 
trieve media with music by melody. 
[001 3) A preferred embodiment of the invention will 
now be described by way of example only and with ref- 
erence to the accompanying drawings, in which: 

FIG. 1 is a histogram of the relative-pitch values in 
a melody retrieval system with thresholds deter- 
mined by using distribution of relative-pitch values 
of notes extracted from a database, wherein thedis- 
trrbuted three categories of U(p), E(qual) and D 
(own) for pitch determining the thresholds are illus- 
trated; 

FIG. 2 is a histogram of the relative-span values in 
the melody retrieval system with thresholds deter- 
mined by using distribution of relative-span values 
of notes extracted from a database, wherein the dis- 
tnbuted three categories of L(onger), E(qual) and S 
(horter) for span determining the thresholds are il- 
lustrated; 

FIG. 3 is a flow chart illustrating the operation of the 
melody retrieval system, and 
FIG. 4 is a melody retrieval terminal device which 
may be used with the melody retrieval system. 

[0014] Referring now to the accompanying drawings 
in which like numerals designate like parts throughout 
the several views thereof, there is shown in FIG. 3, a 
first embodiment of a melody retrieval system. 
[0015] In preparing for retrieval, sequences of rela- 
trve-pitch difference and relative-span ratio are first ob- 
tained from sequences of pitch and span values of each 
piece of music in a database (S101), and a histogram 
of the relative-pitch values and a histogram of the rela- 
trve-span values of the database are made by using the 
distribution of all notes in the database (S102). 
[0016] In FIG. t , the relative-pitch value is the normal- 
ized pitch difference between the adjacent notes in 
which a semitone difference is normalized to the value 
of 100. 

[001 7] In FIG. 2, the relative-span value is expressed 
by percentage. 

[0018] in making these histograms, the total frequen- 
cy of values of a histogram of pitch is expressed by 
Sum1, and that of a histogram of span is expressed by 
Sum2, respectively. The number of categories for the 
histogram of relative-pitch values is expressed by 
Category__Num1, and that of relative-span values is ex- 
pressed by Category_Num2 respectively. 
[0019] In this case, thresholds for relative-pitch val- 
ues are determined so that each category, which will be 
divided by the thresholds, equally has M1 (=Sum1 I 
Category_Num1) values in it. Thresholds for relative- 
span values are similarly determined so that each cat- 



egory equally has M2 (=Sum2/ Category_Num2)va\i>es 
in it (S103). Subsequently the relative-pitch difference 
and relative-span ratio are respectively converted into 
the approximate relative-pitch values and relative-span 
5 values by categorizing those values in several parts like 
U, E, D in FIG. 1 or S, E, Lin FIG. 2 which are separated 
by the thresholds previously calculated (S104). 
[0020] Each number of category is 3 in FIGS. 1 and 2. 
[0021] A song is inputted by a user singing, humming, 
*o whistling or similar into a microphone 1 0 as preparation 
for retrieval (S105). The song is subjectedtoan A/Dcon- 
version (S106), and the A/D converted signal is written 
into a memory 16, through which the A/D converted sig- 
nal is processed by a processor 14 in accordance with 
« a program of the memory 16 to detect a voiced sound 
from the A/D conversion signal (S107). A fundamental 
frequency of each frame is defined from the detected 
voiced sound (Si 08). 

[0022] An onset time of the voiced sound is divided 
20 off as an onset time of each note, a time difference 
(number of frames) with an onset time of the next note 
is determined as the span of the note, and the maximum 
value among the fundamental frequencies of each note 
contained during its span is defined as the highest pitch 
25 values (S109). 

[0023] The relative pitch and span values (S110) of 
each note in the input song are calculated from the ob- 
tained pitch and span from the previous note. The rela- 
tive values are then transmitted to a song retrieval sys- 
30 tem A . ar *l this transmittal may take place over the 
WWW network. 

[0024] As shown in FIGS. 3 and 4, the relative-pitch 
values and relative-span values of the user's input mel- 
ody are respectively converted into approximate rela- 
ys tive-pitch values and approximate relative-span values 
by using the same thresholdsof (S1 04) and categorizing 
relative-pitch values and relative-span values in several 
parts like U, E, D in FIG. 1 or S, E, L in FIG. 2 (S111). 
These approximate values are compared with the value 
to of each melody in the music database, a distance be- 
tween the pitch and span of each melody in the music 
database and the input is calculated (S112), and the ti- 
tles of pieces of music, which are sorted by the order of 
short distance previously calculated, are shown on a 
4$ display 20 of an input device B (S 1 1 3). 

[0025] As particularly shown in FIG. 4, the input de- 
vice B comprises a microphone 10, an A/D converter 
12, a processor 14, a memory 16, an input and output 
unit 18, a display 20, a speaker 22 and a D/A converter 
so 24. 

[0026] It can be understood from FIG. 4 that the input 
and output unit 18 may be connected to a WWW net- 
work (not shown). 

[0027] Received from the song retrieval system A is 
55 the retrieved result, which is shown in the display 20, 
and the desired melody among the retrieved songs can 
be heard from the speaker 22. 

[0028] As described in the foregoing paragraphs, the 
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,i,le ol the desired song can be easily and accurately 
retrieved by inputting the melody by singing into the mi- 
crophone 10. 



Claims 

1 . A melody retrieval method, comprising the steps ol: 
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obtaining sequences ot relative-pitch differ- 
ence and relative-span ratio Irom sequences ot 
pitch and span values ot each piece of music 
in a database; 

making a histogram of the relative-pitch values 
of the database by using the distribution of all 
notes in the entire database; 
making a histogram of the relative-span values 
of the database by using the distribution of all 
notes in the entire database; 
determining thresholds for relative pitch so that 
each category which will be divided by the 
thresholds, equally having M1 (=Sum1 I 
Category_Num1) values in it. and thresholds 
for relative-span values are similarly deter- 
mined so that each category equally has M2 
(=Sum2l Category_Num2) values in it, 
converting the relative-pitch difference and rel- 
ative-span ratio into the approximate relative- 
pitch values and approximate relative-span val- 
ues respectively; 
inputting a song by humming, singing or similar 
into a microphone as preparation for retrieval: 
subjecting the inputted song to an A/D conver- 
5 ion* 

detecting a voiced sound from a sound signal 
after A/D conversion, 

defining a fundamental frequency of each 
frame from the detected voiced sound; 
dividing off an onset time of the voiced sound 
as an onset time of each note; 
determining a time difference (number of 
frames) as the span of the note; 
defining the maximum value among the funda- 
mental frequencies ot each note contained dur- 
ing its span as the highest pitch value; 
calculating the relative pitch and span values 
from the obtained pitch and span from the pre- 
vious sound; 

converting the relative pitch and span informa- 
tion of the user's input melody into approximate 
relative-pitch values and approximate relative- 
span values by using the thresholds previously 
obtained; 

comparing the approximate relative-pitch val- 
. ues and approximate relative-span values with 
the approximate relative-pitch values and ap- 
proximate relative-span values of each melody 
in the music database; 
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calculating a distance between the pitch and 
span of each melody in the music database and 
the input; 

retrieving at least one piece of music based on 
the calculated distances; and 
presenting information about the retrieved mu- 
sic. 

> A melody retrieval method as claimed in claim 1, 
wherein the steps concerning the database and the 
comparison of the coarse values are carried out at 
a first location, and said steps concerning the 
processing of the input song take place at a second 
location, the relative pitch and span values of the 
input song being transmitted from said second lo- 
cation to said first location, and the information 
about the retrieved music being transmitted from 
saidtirst location to said second location, preferably 
via the World Wide Web. 

3. A melody retrieval method as claimed in any pre- 
ceding claim, further including a step of. 

retrieving a piece of media with music by mel- 
ody retrieval. 

4. A melody retrieval system, comprising: 

a database of pieces ot music, 
first means for obtaining sequences ol relative- 
pitch difference and relative-span ratio from se- 
quences of pitch and span values of each piece 
of music, making a histogram of the relative- 
pitch values of the database by using the dis- 
tribution of all notes in the entire database, 
making a histogram of the relative-span values 
of the database by using the distribution of all 
notes in the entire database, determining 
thresholds for relative pitch so that each cate- 
gory which will be divided by the thresholds, 
equally having M1 (=Sum1 1 Category_Num1 ) 
values in it. and thresholds for relative-span 
values are similarly determined so that each 
category equally has M2 (=Sum2 I 
Categoiy_Num2) values in it. and converting 
the relative-pitch difference and relative-span 
ratio into the approximate relative-pitch values 
and approximate relative-span values respec- 
tively; 

input means for inputting a song, 
processing means for subjecting the inputted 
song to an A/D conversion, detecting a voiced 
sound from a sound signal after A/D conver- 
sion defining a fundamental frequency of each 
frame from the detected voiced sound, dividing 
off an onset time of the voiced sound as an on- 
set time of each note, determining a time differ- 
ence (number of frames) as the span of the 
note, defining the maxrnum value among the 
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undamenta, frequencies of each no.e con- 
a-ned during its span as «he highest pj tc h val- 
ue, and calculating the relative pitch and span 
values from the obtained pitch and span from 
the previous sound; 

second means lor convening the relative pitch 
and span information of the user's input melody 
•nto approxmate relative-pitch values and ap 

ZZZZL rGla,iVe - Span Va,ues * using the 
thresholds previously obtained, comparing the 
approximate relative-pitch values and approxi- 
mate relative-span values with the approximate 
rebt,ve-p„ch values and approximate relative- 
span values of each melody in the music data- 
base, calculating a distance between the pitch 
andspanof each melody ,n the music database 
and the ,npui. and retrieving a. .east one p,ece 
of muse based on the calculated distances; 

abounS 10 ", '°' PreS6n,in 9 

about the retrieved music. 

A melody retrieval system as claimed in claim 4 
"herein sa.d fi rs , ^ and saj(J 4 q 
-ans are jn d)(ferent fjfs( ess-ng 

said processmg means being connected to a.tow 
^ansm.ssion of information therebetween prefeTa 
bly via the World Wide Web. 
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retrieving at least one piece of music from the 
database based on the results of the compari- 
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A melody retrieval system comprising; 

a database of pieces of music; 
«rst means for analysing each piece of music 
•n a database of pieces of music and categoriz- 
ing each piece of music according to the pitch 
and span of its notes; 

second means for analysing an input piece of 
music and categorizing it according to the pitch 
and span of its notes; 

third means for comparing the categorized in- 
put piece of music to the categorized pieces of 
music in the database, and 
fourth means for retrieving at least one piece of 
music from the database based on the resutts 
of the comparison. 

10. Computer software on a carrier, allowing any of the 
said means in claim 9 to cany out the actions spec- 
ified m that claim. 
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claim 6. where.n said input means, said procession 
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Computer software on a carrier allowing any of said 
firs, means said processing means and said sec 

A melody retrieval method comprising the steps of: 

analysing each piece of music in a database of 
Pieces of muse and categorizing each piece of 
muse according to the pitch and span of its 

analysing an input piece of music and cateqo- 
nzing ,t according to the pitch and span of its 

comparing the categonzed input piece of music 
«o the categorized pieces of music in the data- 
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